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Abstract 

Let A be a standard Borel space, and consider the space A N(k) of A-valued 
arrays indexed by all size-fc subsets of N. This paper concerns random measures 
on such a space whose laws are invariant under the natural action of permutations 
of N. The main result is a representation theorem for such 'exchangeable' random 
measures, obtained using the classical representation theorems for exchangeable 
arrays due to de Finetti, Hoover, Aldous and Kallenberg. 

After proving this representation, two applications of exchangeable random 
measures are given. The first is a short new proof of the Dovbysh-Sudakov Rep- 
resentation Theorem for exchangeable PSD matrices. The second is in the formu- 
lation of a natural class of limit objects for dilute mean-field spin glass models, 
retaining more information than just the limiting Gram-de Finetti matrix used in 
the study of the Sherrington-Kirkpatrick model. 

Resume 

Soit A un espace de Borel standard, et soit j4 n< ' l'ensemble des tableaux a 
valeur dans A indexes par les sous-ensembles de N de taille k. On s'interesse aux 
mesures aleatoires sur un tel espace dont la loi est invariante par Taction naturelle 
des permutations de N. Le resultat principal est une representation de ces mesures 
aleatoires "echangeables", obtenue a partir des theoremes de representations clas- 
siques de de Finetti, Hoover, Aldous et Kallenberg pour des tableaux echangeables. 

Apres avoir prouve cette representation, on en donne deux applications. La 
premiere est une nouvelle courte preuve du fheoreme de representation de Dovbysh- 
Sudakov pour des matrices PSD echangeables. La seconde concerne la formula- 
tion d'une classe naturelle d'objets limites pour des modeles de champs moyens 
dilues pour des verres de spins qui capture plus d' information que la seule matrice 
limite de Gram-de Finetti qui est notamment utilisee dans 1' etude du modele de 
Sherrington-Kirkpatrick. 
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1 Introduction 



The theory of exchangeable arrays of random variables emerged in work of Hoover ifTUl 
[TTIl . Aldous IHIEIO and Kallenberg lfT2l[T3l . and amounts to a significant generaliza- 
tion of the classical de Finetti-Hewitt-Savage Theorem on exchangeable sequences. 
The heart of the theory is a collection of representation theorems for general such 
arrays, which then beget more specialized representation results such as the Dovbysh- 
Sudakov Theorem for exchangeable PSD matrices. 

This note will consider the related setting of random measures on spaces of arrays, 
where now the laws of those random measures are assumed invariant under the relevant 
group action. Intuitively, this introduces an 'extra layer of randomness'. In order to 
introduce these formally, let [n] :— {1,2, ... ,n} for n £ N, let S® = (J n>1 St n ] 
be the group of all permutations of N which fix all but finitely many elements, and 
consider a measurable action T : SW rx E on a standard Borel space E. In full, this is 
a measurable function 

T : S N xE — ► E : (a, x) >-> T a x 

such that 

T idN = id B and T ai T a2 x = T <T2<Tl x Voi ,a 2 ,x. 

As is standard, if /i 6 Pr E and a € Sn then T° fi denotes the image measure of /i 
under T a . 

Definition 1.1. If E is a standard Borel space andT : Sfi r\ E is a measurable action, 
then an exchangeable random measure ('ERM') on (E, T) is a random variable fi 
taking values in Pr E such that 

/i'™T> VaeS N ; 

that is, 

H{A) l = n{x : T a x G A} Vcr G S®, A C Bor ci E. 

These are essentially what ergodic theorists call 'quasi-factors' [8 1 Chapter 8]. We 
will study these for the group actions that underly the theory of exchangeable arrays. 
Given a standard Borel space A and k £ N, the space of /c-dimensional arrays valued 
in A is ^4 N<M i where denotes the set of size-A: subsets of N. An element of such 
a space of arrays will often be denoted by (x e )\ e t—k or similarly. (In the following, 
one could focus instead on arrays indexed by ordered /c-tuples, but we have chosen the 
symmetric case as it is a little simpler and arises more often in applications.) The group 
iSn acts on ^4 N<fe) by permuting coordinates in the obvious manner: 

T° '{{x e )\e\=k) = (aV(e))|e|=kj 

where cr(e) — {a(i) : i £ e}. Slightly more generally, our main results will also allow 
Cartesian products of such actions over finitely many different k. Thus, our arrays will 
usually be indexed by the family N(- fe ' of subsets of N of size at most k for some fixed 
k. 
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Examples. (1) If an exchangeable random measure fi on (E, T) is deterministic, then 
its constant value must itself be invariant under the action T. In case E = A Nlk> with 
the action above, this means fi is almost surely the law of an exchangeable ^4-valued, 
pj( fc )-indexed array. 

(2) On the other hand, if p is a T-invariant measure for any action (E, T), then 
another way to obtain an exchangeable random measure from it is to let 

A* := S x 

where X is a random element of E with law /i, and Sx is the Dirac mass at X. 

(3) In case E — A N(k> with the action above, example (2) fits into a more gen- 
eral family as follows. The space of probability measures Pr A is also standard Borel 
with the Borel structure generated by evaluation of measures on Borel sets. Suppose 
(A e ) | e |=fe is an exchangeable array of (Pr ^4)-valued random variables, and now let 

|e|=fc 

This class of examples will feature again later. Such an example is called an exchange- 
able random product measure ('ERPM'). 

(4) It is also easy to exhibit an ERM which is not ERPM. For example, let II = 
(A, B) be a uniform random bipartition of N (this is obviously exchangeable), and 
having chosen IT let /x G Pr{0, 1} N<2) be the probability which has two atoms of mass 
i on the points 

1 a n( 2 ) an d 1 b n(2) • 

(5) Lastly, given a measurable family of exchangeable random measures fi t in- 
dexed by a parameter t G [0, 1), we may average over this parameter to obtain a mixture 
of these exchangeable random measures: 

H= [ n t dt. 
Jo 

This is clearly still exchangeable. < 

The main result of this paper characterizes all ERMs on spaces of arrays. To moti- 
vate it, we next recall the Representation Theorem for exchangeable arrays themselves. 
This requires some more notation. 

First, for any set S we let VS denote the power set of S. 

Next, suppose that B a , B\, . . . , B k and A are standard Borel spaces. A Borel 
function 

/ : B Q x B\ x B [ f 2) x • • • x B k = JJ £?f ]W — ► A 

i<k 

is middle-symmetric if 

f(x, (ajj)i e [fc], {x a )ae[k\V) , • • - ,X[k]) = f( x i { x <j(i))ie[k]i { x a(a))ae[k]W ) • • • : x [fc]) 

for all a G . 
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Given standard Borel spaces Bq, B\, . . . , Bk and A$, A\, . . . , Ak, and middle- 
symmetric Borel functions 

/ J J />* >.l,. i = l,...,k, 

j<i 

we will write / for the function 

J]sf] (i) -> n^ 1<0 : W«W ^ (/|e|((^)aCe)) eg[fe] , 

which combines all of the /j. 

The tuple (/o, . . . , fk) is referred to as a skew-product tuple, and the associated 
function / as a function of skew-product type; clearly the latter determines the former 
uniquely. 

Example. If k = 2, then a function of skew-product type [0, — > [0,1) takes the 
form 

f(x,X 1 ,X 2 ,X 12 ) = (fo(x),fl(x,X 1 ),f 1 (x,X2),f2(x,X 1 ,X2,X 12 )). 

< 

It is easily checked that if / and g are functions of skew -product type for the same k, 
then so is go f. In terms of (/o, . . . , fk) and (go, ... , gk) this composition corresponds 
to the skew-product tuple 

hi((Xa) a C[i]) ■= 9i((f\a\{{xb)bCa)) am ), t = 0, 1, . . . , fc. 

Slightly abusively, we will also write / for the related function 

Y[Bf } ^Y[Af ] : (» e )| e |< fc -> (/|e|((*«)aCe)) |e| < fc , 

which also determines (/o, . . . , /&) uniquely. 

Theorem 1.2 (Representation Theorem for Exchangeable Arrays; Theorem 7.22 in [ 15 1). 

Suppose that Aq, A\, Ak are standard Borel spaces and that (X e )| e |<fe is an ex- 
changeable random array of r.v.s with each X e valued in A\ e \. Then there are middle- 
symmetric Borel functions 

fi '■ [0, 1) V ^ — > Ai, i = 0,l,...,k, 

such that 

{X e )\ e \<k = (/|e|((^o)oCe))| e |< fc = / ((^e)|e|<fc)> 

where (f7 e )|e|<fc is an i-i-d. family o/U[0, l)-r.v.s. □ 
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The companion Equivalence Theorem, which addresses the non-uniqueness of the 
representing function /, will be recalled later. 

To produce a random measure, the idea will simply be to use directing functions ft 
that depend on two sources of randomness, and then condition on one of them. 

Theorem A Suppose that yL is an ERM on Aq x ■ ■ ■ x ' . Then there are middle- 
symmetric Borel functions 

fi : ([0, 1) x [0, 1))^! — ► Ai 

such that 

/*(■) '= P(/((f / e,Ve)| e |<fe) G ■ | (C/ e )| e |< fe ), 

where U e and V e for e C N, |e| < k are all i.i.d. ~ U[0, 1). On the right-hand side, 
this is a measure-valued random variable as a function of the r.v.s (Ue)\ e \<k- 

We will find that after some manipulation of the problem, Theorem A can be de- 
duced from the Representation Theorem and Equivalence Theorems for exchangeable 
random arrays themselves. 

The proof of Theorem A can be considerably simplified when k = 1, so we will first 
prove that case separately. In that case, the structure given by Theorem A is essentially 
a combination of examples (3) and (5) above. To see this, we reformulate the result as 
follows. 

Given a standard Borel space A, let B([0, 1), Pr A) denote the space of Lebesgue- 
a.e. equivalence classes of measurable functions [0, 1) — > Pi A. Then_B([0, l),PrA) 
has a natural measurable structure generated by the functionals 

f^f <(>(t)f(t,B)dt 
Jo 

corresponding to all <fi e L°° [0,1) and Borel subsets B C A. This measurable structure 
is also standard Borel: for instance, if one realizes A as a Borel subset of a compact 
metric space, then the above becomes the Borel structure of the topology of conver- 
gence in probability on B([0, 1), Pr A), which is Polish. 

Theorem B If fi is an ERM on A N , then there is an exchangeable sequence of r.v.s 
(Aj)jeN taking values in B([0, 1), Pr A) such that 

M(-) l = f (®Ai(t,-))dt. 
So when k = 1, every ERM is a mixture of ERPMs. 

With the structure given by Theorem B, one may next apply the de Finetti-Hewitt- 
Savage Theorem to the sequence Ai to obtain a random measure 7 on B([0, 1), Pr A) 
such that Xi is obtained by first choosing 7 and then choosing Aj i.i.d. with law 7. We 
write Samp(7) for the ERM obtained by this procedure, and refer to 7 as a directing 
random measure for fi. 



5 



After proving Theorems A and B, we offer a couple of applications of the case 
k = 1. These applications can also be given higher-dimensional extensions using the 
cases k > 2, but those extensions seem less natural. The reader interested only in the 
applications need not read the proof of the general case of Theorem A. 

The first application is a new proof of the classical Dovbysh-Sudakov Theorem: 

Dovbysh-Sudakov Theorem Suppose (Rij)ij£N is a random matrix which is a.s. 
positive semi-definite, and is exchangeable in the sense that 

{R<j{i)cr(j))i,j — {Ri'j)i,j V(T G /SfJ. 

Then there are a separable real Hilbert space S) and an exchangeable sequence ai)j £ N 
of random variables valued in x [0, oo) such that 

(Rij)i,j = + Sijai)ij, 

where &y is the Kronecker delta. 

This first appeared in [7], and more complete accounts were given in [9] and 1 18|. 
The proofs of Hestir and Panchenko start with the Aldous-Hoover Representation The- 
orem, which treats (Rij)ij as a general two-dimensional exchangeable array. They 
then require several further steps to show that the PSD assumption implies a simplifi- 
cation of that general Aldous-Hoover representation into the form promised above. On 
the other hand, we will find that if one simply interprets {Rij)i,j as the covariance ma- 
trix of an exchangeable random measure, then one can read off the Dovbysh-Sudakov 
Theorem from Theorem B, which in turn does not require the Aldous-Hoover Theorem. 

Our second application is to the study of certain mean-field spin glass models, and 
particularly Viana and Bray's dilute version of the Sherrington-Kirkpatrick model (22). 
In the case of the original Sherrington-Kirkpatrick model a great deal has now been 
proven, much of it relying on the notions of 'random overlap structures' and their di- 
recting random Hilbert space measures: see, for instance, Panchenko's monograph |[T9l . 
The analogous theory for dilute models is less advanced. In this note we will simply 
sketch how the main conjecture of Replica Symmetry Breaking can be formulated quite 
neatly in terms of limits of exchangeable random measures, translating from the earlier 
works [20 17]. We will not recall most of the spin glass theory behind this conjecture, 
but will refer the reader to those references for more background. 

Acknowledgements 

1 am grateful to Kavita Ramanan for some helpful references, and to Dmitry Panchenko 
for several valuable suggestions. 

2 The replica trick 

The key to Theorem A is the following simple observation. It is essentially an abstract 
version of the 'replica trick' from statistical physics ([16]), and has appeared in the 
more general setting of representing quasi-factors in ergodic theory ([8, Chapter 8]). 



6 



Before we proceed, first observe that, since any standard Borel space is isomor- 
phic to a Borel subset of a compact metric space, we may replace the spaces Aq, . . . , 
Ak with such enveloping compact spaces in Theorems A and B, and so assume these 
spaces are themselves compact. We will make this assumption throughout the proofs 
of those theorems, although some non-compact examples will re-appear later in the 
applications. 

Proposition 2.1 (Replica trick). Ifn is an ERMon \\ i<k Af , then there are auxiliary 

standard Borel spaces Aq, A\, A k and an exchangeable array (y^, -X" e )|e|<fc °f 
random variables such that 

• each (Y e , X e ) takes values in Ai e i x Ai e i, and 



Proof. After enlarging the background probability space if necessary, we may couple 
the random variable fi with a doubly-indexed family of random variables 



all taking values in one of the AiS, as follows: 

• first, sample the random measure fi itself; 

• then, choose the sub-families (X e )| e |< fe , (Xi ie )| e |< fc , (X2,e)\e\<k> ■ ■ - independently 
with law fi. 

In notation, this coupling is defined by 

P(PQ| e |<fc G da, (X lje )| e |< fc G da x , {X 2 , e )\ e \<k 6 da 2 , ... | /x) 



Having done this, let Ai :— Af and let Y e :— (Xj, e )j e fi S A| e | for each e G 
psj(< fc ) xhe exchangeability of implies that the joint distribution of the family ([TJ 
is invariant under applying elements of S*n to the indexing sets e, and hence that the 
process (Y e , X e )\ e \ <k is exchangeable. 

On the other hand, since we assume each At is compact, so is Yii<k ^? ' > an d now 
the Law or Large Numbers shows that in the above process one has the a.s. convergence 
of empirical measures 



• one has 



m(-) - w P((x e )| e |< fc e-|(^ e |)| e |< fc ). 




(i) 



fi(da) ■ /x(dai) • /u(da 2 ) . 




n=l 



in the vague topology on Pr Ili<fc -^i 
Therefore in the process 
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the family of r.v.s (Y^)| e |<fc determine /j, a.s., whereas conditionally on /j, the family 
(y e )|e|<fe becomes independent from (X e )| e |<fc- This implies that 

P((X e )| e <fc € • | (Y e )\ e \<k) = P((^e)|e<fc 6 " | A») = /*(') a - S - 



as required. 



□ 



3 Proofs in one dimension 
3.1 Some preliminaries 

We will repeatedly need the following standard tool from measure-theoretic probabil- 
ity. See, for instance, the slightly-stronger Theorem 6.10 in [ 14] . 

Lemma 3.1 (Noise-Outsourcing Lemma). Suppose that A and B are standard Borel 
spaces and that (X, Y) is an (A x B)-valued r.v. Then, possibly after enlarging the 
background probability space, there are a r.v. U ~ U[0, 1) coupled with X and Y and 
a Borel function f : A X [0, 1) — > Y such that U is independent from X and 



Of course, the function / in this lemma is highly non-unique. The degenerate case 
in which X is deterministic is still important: it reduces to the assertion that for any 
standard Borel probability space (B, v) there is a Borel function / : [0,1) — > B such 
that /([/) ~ v when U ~ U[0, 1). 

Finally, let us recall the full de Finetti-Hewitt-Savage Theorem for the case k = 1, 
which is rather stronger than just the case k = 1 of Theorem ll.2l The following is the 
combination of Proposition 1.4 and Corollaries 1.5 and 1.6 in 031 . 

Theorem 3.2. Suppose A is a compact metric space and (X n ) n is an exchangeable 
sequence of A-valued r.v.s. Then the sequence of empirical distributions 



converges a.s. to a (Pr A)-valued r.v. W which has the following properties: 

(i) W is a.s. a function of (X n ) n , 

(ii) W generates the tail cr-algebra of (X n ) n up to /^-negligible sets; 
(Hi) the r.v.s X n are conditionally i.i.d. given W; 

(iv) if Z is any other r.v. on the same probability space such that 



(X,Y) = {XJ(X,U)) a.s.. 



□ 




N 



n=l 



[Z,X\,X 2 , ...) = (Z,X a ^,X a ^ 2 ), . . .) Ver e Spj, 



then Z is conditionally independent from (X n ) n over W. 



□ 
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3.2 Proofs in one dimension 

Proof of Theorem A in one dimension. Suppose /j, is an ERM on A N and let (Y n , X n ) n 
be a process as given by Proposition 12. 11 

We next apply Theorem 13. 21 twice: first to the sequence (Y n ) n , to obtain a r.v. W 
taking values in E ;= Pr A; and secondly to (Y n , X n ) n , to obtain a r.v. Z taking values 
in F := Pr (A x A). From their definitions as limits of empirical distributions, W is 
almost surely a function of Z. On the other hand, property (iv) of Theorem 13 . 2 1 gives 
that Z is conditionally independent from (Y n ) n over W. 

Now pick ann 6 N. By Lemma [3~T1 there is a Borel function /i : E x F x A x 
[0, 1) — > A such that 

(w, z, y„, x„) '= (w, z, y„, z, Y n , v n )), 

where (V n ) n are i.i.d. ~ U[0, 1) and are independent from (W, Z, Y n ). Moreover, this 
same fi works for every n, by exchangeability. It follows that in fact 

(W,Z,(Y n ,X n ) neN ) (W,Z,{Y n ,h{W,Z,Y n ,V n )) nm ), (2) 

because both sides have the same marginals for individual n, and both sides are condi- 
tionally i.i.d. over (W, Z), so all finite-dimensional marginals agree. 

Next, another appeal to Lemma |3~T1 gives a Borel function g : Ex [0, 1) — > F 
such that 

(W,Z) la ^(W,g(W,V)) 
with a new independent V ~ U[0, 1). This implies that 

(W, Z, (Y n ) n6N ) = w (W,g(W, V), (y„) n6N ), 

because Z is conditionally independent from (Y n ) n over VF, so again all finite-dimensional 
marginals agree. Combining this with (fJJ gives 

{w,z, (Y n ,x n ) neN ) '= (w, 5 (w,y),(y n ,/ 2 (w,F„,y,F n )) neN ), 

where 

f 2 (w,y,v,v / ) := fx{w,g{w,v),y,v'). 

It follows that 

p(PQ„ g • | (y„)„) = p((/ 2 (w,y n ,v;y n ) n ) n g • | (r„) n ). 

Finally, we may apply de Finetti's Theorem again, this time to (Y n ) n , to obtain Borel 
functions hi : [0, 1) — > E and h 2 : [0, l) 2 — > A such that 

(W,(y n ) neN ) l = (/ll(f/),(/l2(C/,C/n))neN), 

where C7 and ([/„)„ are i.i.d. ~ U[0, 1) r.v.s, independent of everything else. Letting 

/(it, u', u, i/) := f 2 {hi(u),h 2 {u, it'), u, u')> 
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substituting for (W, (Y n ) n ) in the above gives 

P((-X»)„ G ' I (Vn)n) '= P((/(^^n,V,F„) n )„ G • | (7, ([/„)„), 
as required. 



□ 



Proof of Theorem B. By the Law of Iterated Conditional Expectation, the representa- 
tion obtained above may be re-written as 



M(0 



law 



= E 



e(p((/(C/, U n , V, V n )n)n G • | f/, ([/„)„, V) | (7, ([/„)„) 
P (/([/, [/„, V, K) G • | U, U n , V) | U, (tT n ) n ) 



iA n (i, - ))d* s 



where 



A„(i, •) := P(/(t/,C/„,i,y„) 



As functions of (U, U n ) for each n, these form an exchangeable sequence of r.v.s taking 
values in B([0, 1), Pr A), so the proof is complete. □ 



3.3 Relation to row-column exchangeability 

A relative of exchangeability for a two-dimensional random array (X;, n )(i,n)eN 2 is 
row-column exchangeability, which asserts that 

(^ CT (i),T(n))i,ra = (^i,n)i,n V(T, T G ^N- 

Since cr and r may be chosen separately, this is a rather stronger symmetry than ordi- 
nary two-dimensional exchangeability. Here, too, there is a representation theorem due 
to Aldous and Hoover, and also a version in arbitrary dimensions due to Kallenberg, 
who calls such arrays 'separately exchangeable'. 

Theorem 3.3 (Corollary 7.23 in |fT31 ). If (-X~i >n )j in is an A-valued row-column ex- 
changeable array then there is a Borel function [0, l) 4 — > A such that 

(f(Z, Ui, V n , Wi, n ))i, n , 

where Z, Uifor i G N, V n for n G N and Wi, n for (i, n) G N 2 are i.i.d. ~ U[0, 1). □ 

An alternative proof of Theorem B can be given via Theorem 13.31 One begins 
with the construction of the two-dimensional random array (Xj jn )j jn as in the proof 
of Proposition 12.11 (where the sets e have become singletons n). Since this array is 
row-column exchangeable, the Representation Theorem gives 

(*<,n)i,n '= (f(U, Ui, V n , Wi, n ))i, n 
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for some Borel directing function / : [0, l) 4 — > A, where U, Ui for i £ N, V n for 
n £ N and Wi, n for i,n £ N are i.i.d. ~ U[0, 1). One can now read off a directing 
random measure y(U) on B([0, 1), PrA), a function of [7 ~ U[0, 1), in the following 
two steps: 

• first, for each fixed U and U' one obtains an element X(U,U') £ £? ( [0 , 1 ) , Pr A) 
according to 

\(u,u')(t, ■) = p w (f(u,u',t,w)€-), w~u[o,i); 

• second, 7(t7) is the distribution of A(t7, {/') where [/' ~ U[0, 1). 

On the other hand, a couple of simple applications of the Noise-Outsourcing Lemma 
show that any directing random measure 7 on B([0, 1), Pr A) can be represented this 
way, so this gives a bijective correspondence 

{directing random measures on B([0, 1), Pr A) up to equivalence} 

O {directing functions [0, l) 4 — > A up to equivalence}, 

where 'up to equivalence' refers to the possibility that different directing random mea- 
sures or directing functions may give rise to the same row-column exchangeable array. 

This approach is the basis of the paper 0171 , to be discussed later. It is quick, but at 
the expense of assuming Theorem l3.3l 

On the other hand, our approach to Theorem B does not use any exchangeability 
theory in dimensions greater than one. Moreover, one can reverse the idea above to 
give a fairly quick proof of Theorem |3.3| using Theorem B. 

Proof of Theorem \3.3\f rom Theorem B. First let (Xj jn )j !n = ((-Xi )n )j)„, thought of as 
an exchangeable sequence of A N -valued r.v.s. By the de Finetti-Hewitt-Savage The- 
orem applied to the exchangeability in n, its law is a mixture of product measures; 
equivalently, there is a (Pr A N )-valued r.v. fi such that 

Iaw((^ i)B )i) n ) = E(/^ N ). 
On the other hand, for any a £ SW the exchangeability in i gives 

E(/0 = law((X iin ),)n) = lM(X*(i),n)i)n) = E((T»^), 

where T a : A n — > A n is the corresponding coordinate-permuting transformation. By 
the uniqueness of the de Finetti-Hewitt-Savage decomposition, this implies 

= Till Ver £ 5 N , 
so /jl is an ERM. Therefore Theorem B gives 

law((X i>n )i) n ) = El / (g)A,(t,.)dtJ 
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for some exchangeable random sequence (Ai)j taking values in B([0, 1), Pr A). 

Next, applying the de Finetti-Hewitt-Savage Theorem to the sequence (Aj)j itself 
gives a Borel function F : [0, l) 2 — > B([0, 1), Pr A) such that the above becomes 

]&w((Xi, n )i) n ) = E z>{Ui)i ( [ (g)F(Z,Ui,t,-)dt) m 

= E z , mi ,(v n )n (g) HZ,Ui,V n ,-), (3) 

(i,n)eN 2 

where Z, Ui for i £ N and y„ for n € N are i.i.d. ~ U[0, 1), and in the second equality 
we have simply changed notation from ' J Q ■ df to 'Ey/. 

Finally, by Lemma [3~T| there is a Borel function / : [0, l) 4 — > A such that 

P(f(Z, U,V,W) e -| Z, C/, V) = [/, V, •) a.s. 

when Z, [/, V, W are i.i.d. ~ U[0, 1), and now the right-hand side of (O becomes 
law((/(J7, Ui, V n ,Wi^)i) n ), as required. □ 



4 Proof in higher dimensions 

Not all parts of Theorem 13.21 generalize to higher-dimensional arrays, and instead we 
must make a more careful argument using the Equivalence Theorem |4. 1 I below. 

4.1 Some more preliminaries 

The Equivalence Theorem characterizes when two functions direct the same process in 
the setting of Theorem ll.2l Its formulation needs the following notion. Let C = [0, l) r 
and D — [0, l) 3 for some r, s € N. Then a skew-product tuple (/q, . . . , fk) from 
CfP[k] to £fP[k\ j s Lebesgue-measure-preserving if for all i = 0, 1, . . . , k and all 

(x a ) a c [{] 6 C W\M one has 

t/~U(C) =► /i((a!a) 5 M ,^)~U(D). 
The Equivalence Theorem is as follows. 

Theorem 4.1 (Equivalence Theorem for directing functions; Theorem 7.28 in lfT31 ). If 

f, f : [0, l)^^! — s> Ao X • • • X Afe are functions of skew-product type such that 

(/|e|((^o)oCe))| e |< fc = (/|e|(( C/ a)aCe)) |e| < fe , 

then there are Lebesgue-measure-preserving functions G,G' : [0, l) 7 ^^"] — > [0, 
of skew-product type which make the following diagram commute: 
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[0,1)^1 




A x A\ x • • • x A k . 



In connection with this theorem, we will also need the following 'factorization' 
result. 

Corollary 4.2. LetUc[k] = (U e ) e c[k] andVg[k] be independent uniform r.v.s valued 
in (>. h ri . If 

G: [0,1)™ — > [0,1)™ 

is a function of skew-product type and Lebesgue-measure-preserving, then there is an- 
other function 

H : ([0,1) x [0,1))™ — > [0,l) p [ fc ] 
of skew-product type and Lebesgue-measure-preserving such that 

Uc[k]=G(H(U m ,Vc[k])) a.s.. 

Another way to express this is that the maps in the following diagram are Lebesgue- 
measure-preserving and a.s. commute: 

([0, 1) x [0, l)) v ^ £ ^ [0, 1)^1 




G 



where 

n((ar e ,J/ e )eC[fc]) = (Xe)eC[k\ 

is the obvious projection. 

Geometrically, the intuition here is that G is 'almost onto' (since its image measure 
is Lebesgue), and that as a result one can represent it as the projection map IT after 
using H to 'straighten out the fibres'. 

Proof. Let G be defined by the skew-product tuple (Go, . . . , Gfc). We must construct 
the skew-product tuple (Ho, ...,Hk) that defines H. In terms of these tuples, our 
requirement is that 

Gi((H lel ((U a ,V a ) aQe )) em )=U [{l a.s. Vi = 0,l,...,fc. (4) 
When i = this simplifies to 

Go(H (U a ,V a )) = U a a.s.. 
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We can obtain such an Ho from the Noise-Outsourcing Lemma IXTl as follows. Let Zq 
be a U[0, l)-r.v. and let Xq := Gq(Zq), so this is also ~ U[0, 1). Applying Lemma [3TTl 
to the pair (Xq, Z ) gives a Borel function H : [0, 1) x [0, 1) — > [0, 1) such that 

(X ,Z ) = (X ,H (X ,Y )) a.s. 

for some Yq ~ U[0, 1) independent from Xq- Since Xq = Gq(Zq), applying Go to the 
second coordinates here gives 

X = G (H (X ,Y )) a.s. 

The general case now follows by induction on i. Suppose that i > 1, let Y e for 
e ^ [i] and Z e for e C [i] be i.i.d. ~ U[0, 1); define X e := G\ e \((Z a ) a c e ) for 
all e C [i]; and assume that Hq, . . . , Hi-i have already been constructed such that 
Z e := Hi e \((X a , Y a ) a ce) for eacn e S [£]• Applying Lemma [XT] again gives a Borel 
function H t : ([0, 1) x [0, — ► [0, 1) and a r.v. Y A ~ U[0, 1) such that 

((X e ) e c[i],(Y e ) e c[ t ],Z [l] ) = ((X e ) e c[H,(Y e ) e C[{\,H i ((X e ) e o[i],(Y e ) e c[i])), 
and as before this is equivalent to the desired equality (@). □ 

4.2 Completion of the proof 

We need the following enhancement of Proposition ^. II 

Lemma 4.3. If fi is an ERM on Y\ i<k , then there is an exchangeable array 
(U e , X e )i e i<fe such that 

• (Ue)\ e \<k are i.i.d. ~ U[0, 1), 

• each X e takes values in ^4| e |> 

• and one has 

H(-) l = P((X e )| e |< fc G ■ | (U e ) H < k ). 

Proof. Let (Y e , -ST e ) | e | < fc be the process given by Proposition 12.11 with each (Y e , X e ) 
taking values in A\ e \ x Ai e i. By the Representation Theorem ll.2l applied to (3^)| e |<jt, 

there is a function / : [0, l) v ^ — y Tii<k or " skew-product type such that 

(Y e )\ e \<k = 7{{Ue)\e\<k), 

where (J7 e )|e|<fc i s an i-i-d. ~ U[0, 1) array. 

Now consider the coupling (U e , X e )\ e \ <k whose law is the relatively independent 

product over the condition (F e )|e|<fc = f((U e )\e\<k)- 

P((U e ) e e du, (X e ) e g da) 

= P((U e ) e £ du) ■ P((X e ) e £ da \ (Y e ) e = /(«)). (5) 
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This is exchangeable and has the three desired properties. The exchangeability 
follows because both factors on the right-hand side of (0 are invariant under the action 
of Sn on the indexing set e, by the exchangeability of (U e ) e and (Y e ,X e ) e . The first 
two of the properties listed are obvious, and the third follows from Proposition 12.11 
because the above relative product formula gives 

P((X e ) e e da | (U e ) e =u) = P({X e ) e e da \ f((U e ) e ) = /(«)). 

□ 

Proof of Theorem A. Let the process (U e , X e )\ e \<^ be as in the preceding corol- 
lary. Applying Theorem 1 1.2 1 to this whole process gives functions q : [0,l)' p ^ — > 
[0, l) p [ fc l and /t : [0, 1) V W — ► ]J i<k Af^ of skew-product type such that 

((^e)|e|<fc, W|e|<fc) '= (fl((^)|e|<fc), KK)\*\<*)) , (6) 

where again (^)| e |<fc are i-i-d. ~ U[0, 1). 
For the first coordinates, this reads 

(Ue)\ e \< k l = d((U' e ) ]el < k ). 

Since both input and output are i.i.d. U[0, 1) arrays, we may apply the Equivalence 
Theoreml4~T1to this equality of laws: it gives functions G, G' : [0, l) v ^ — > [0, l) v ^ 
of skew-product type and Lebesgue-measure-preserving which make the following di- 
agram commute: 

[0, 1)^1 



(5 




[0,1) V W. 

Now applying Corollary 14.21 to G gives a Lebesgue-measure-preserving function 
H : ([0,1) x [0, l)) v ^ — y [0, l) v ^ of skew-product type such that the above com- 
mutative diagram can be enlarged to 

([0, 1) x [0, [0, 




[0,l) v[k] - 
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Let (V e )| e |<fe be another collection of i.i.d. U[0, l)-r.v.s independent from {U e )\ e \< k , 
and let / := h o G' o H. Then the above diagram implies that: 

• on the one hand, 

{U e )\e\<k =g(G'(H((U e ,V e ) lel < k ))) a.s., 

• and on the other, G'{H{(U e , V e )\ e \< k )) is an i.i.d. array of U[0, l)-r.v.s, and so 

(MK)H<k)MU' e )\e\<k)) 

1 i, w (5(G'( J ff((t/ e ,K)| e |< fc ))),MG'( J ff((;7 e ,K)| e |< fc ))))- 
Combining §6^ with these two facts now gives 

(U e ,X e ) mk ^ (g(G'(H((U e ,V e )\ e \< k ))),h(G'(H((U e , t4)|e|<fc))))| e |<fc 

= {(U e )\ e \<k, f((U e , V e )\ e \< k )), 

and conditioning both sides of this on (f7 e )|e|<k gives 

P((*e)|e|<* G • | (U e ) M < k ) = P(f((U e ,V e ) [e[ < k ) g • | (f/ e )|e|<fe), 

as required. □ 

5 Relation to Dovbysh-Sudakov Theorem 

Proof of Dovbysh-Sudakov Theorem. The trick to this is the standard one-to-one cor- 
respondence 

{PSD (N x N)-matrices} <H» {Gaussian measures on M N } 

in which a Gaussian measure is identified with its variance-covariance matrix. (This is 
elementary for finite PSD matrices, and then the infinite case follows by the Daniell- 
Kolmogorov Theorem: see lfl4l Theorem 6.14].) Because Gaussian measures are 
uniquely determined by their variance-covariance matrices, this correspondence inter- 
twines the two permutations actions of N, so from (Rij)ij we may construct an ERM 
on M N which is almost surely Gaussian, and such that 

Rij = / XiXj n(d(x n ) ne n) a.s.. 

Now Theorem B gives a representation 

A* = w [ 6dAi(i,-)d* 
Jo l 
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and 



with (Aj)j drawn from some exchangeable sequence taking values in B([0, 1), Prl 
Substituting this above gives 

Ru l = / / x 2 \i(t,dx)dt 
Jo Jr 

Rij l = / ( / x Xi(t, dx)) ( / x\j(t,dx))dt when i ^ j. 
Jo Jr Jr 

Letting 

fi = £ 2 ([0,l),d<), 
&(t) = / a; Ai(t, da;) 



and 

2 N 



( / x 2 X l (t,dx) -H x\i(t,dx)^ ) 



dt, 



this is the desired representation. 

(Note that ^ must be in f) a.s. because 



l 

2 



^ (y a;Ai(i,da;)) 2 di < J J x 2 Xi(t, dx) dt l = i? r 



which is finite a.s.) □ 



6 Limiting behaviour of the Viana-Bray model 

Our second, and much more tentative, application for ERMs is to the study of the 
Viana-Bray ('VB') model [22 1 . This is the basic 'dilute' mean-field spin glass model. 
On the configuration space {— 1, 1}^, it is given by the random Hamiltonian 

M 

H N (a) Jk<Ti k <Tj h , (7) 

fc=i 

where: 

• M is a Poisson r.v. with mean aN (the thermodynamic limit is be taken with a 
fixed); 

• iii ill is, 32, •••are indices from [N] chosen uniformly and independently at 
random; 

• and Ji, J2, ... are i.i.d. symmetric M-valued r.v.s with some given distribution, 
often taken to be uniform ±1. 
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(There are many essentially equivalent variants of this model, but this popular version 
will do here.) From a quenched choice (that is, a fixed sample) of the random function 
Hn, the objects of interest are the resulting Gibbs measure 

1P,nW} = 7 \n\ exp(-PH N (a)), 

Z N(P) 

the partition function 

(7 

and the expected specific free energy 

F N (f3) = ±E\ogZ N (l3), 

where the expectation is over the random function Hn We will sometimes drop the 
subscript '/?' or W in the following. 

This is a relative of the older Sherrington-Kirkpatrick ('SK') model 1 16], in which 
all pairs of spins ij interact according to independent random coefficients ~ N(0, 1 /N ). 
The rigorous study of the SK model has become quite advanced in recent years; we will 
not credit all of the important contributions, but refer the reader to the books lETl [191 
and the many references given there. By contrast, most properties of the VB model 
remain conjectural. 

A key tool in the study of the SK model is the use of random measures on Hilbert 
space as a kind of 'limit object' for the random Gibbs measures 7,3,7V as N — > 00. 
Viewing —={ — 1, 1} N as a subset of 1% , Jp,N is itself a random Hilbert space measure, 
and the appropriate notion of convergence is convergence in distribution of the Gram- 
de Finetti matrices obtained by sampling. This use of limit objects is explained more 
carefully in [ 19 1. The key point is that the main properties of the SK model, such as the 
free energy, really depend only on the covariances among the random variables H(a), 
and hence on this Hilbert space structure. 

This is no longer true for the VB model, so a more refined tool is needed. One 
possibility has been explored in [17], and before that physicists and mathematicians had 
already worked with the related notion of 'multi-overlap structures' (see, e.g., fl5]|6], 
and also [20], although the latter does not use that terminology). Here we will simply 
propose exchangeable random measures as a fairly intuitive equivalent formalism, and 
compare it with two predecessors from the literature: the weighting schemes used by 
Panchenko and Talagrand in lEOl . and Panchenko's use of directing functions in ifTTl . 
After introducing our notion of 'limit object', we will give a fairly brisk summary of 
the translations between these formalisms; the calculations are all routine. We will 
restrict attention to the Viana-Bray model as above for simplicity, but the discussion 
could easily be extended to a more general class of dilute models, as in ll20l[l7l . 

6.1 Basic idea 

If jp t N is as above, then it defines an ERM // by sampling: first quench the random 
measure jp t j?; then select samples (called 'replicas') a , a 2 , ...G {—1,1}" i.i.d. 
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~ 7at; and finally use these to define /x as a mixture of delta masses: 

1 N 

n=l 

Identifying ±1 with the extreme points of Pr{ — 1, 1}, this is clearly a mixture of 
ERPMs of the kind considered previously. Let Samp^^jv) be the law of fi. 

It now makes sense to say that 7^, at sampling converges to some random prob- 
ability measure 7^ on B([0, l),Pr{— 1, 1}) if Samp(7^jv) converges to Samp(7 /3 ) 
for the vague topology on Pr(Pr{— 1, 1} N ). Since this last space is compact, one 
can always at least take subsequential limits of (Samp^^Ar))^, and now Theorem 
B promises the existence of some 7^ that represents the limiting ERM (although it is 
unique only up to equivalence). 

6.2 Comparison with weighting schemes and directing functions 

In ||20| the authors do not introduce a notion of limits as such for the random measures 
70 jv> but they do formulate their most general results (Section 3 of that paper) in terms 
of some data that they call a 'weighting scheme'. This consists of: 

• a sequence of R-valued r.v.s (X k ) k , and a family {(Xu 3 )k)i,j of i.i.d. copies of 
this sequence indexed by G N 2 ; 

• and, independently of these, a [0, 1] -valued random sequence of weights (v k ) k 
such that J2k v k = 1- 

These data appear in an upper-bound formula for the free energy which will be 
recalled below. They can be encapsulated in a certain directing random measure 7 on 
B([0, 1), Pr{— 1, 1}) as follows. First, identifying elements of Pr{— 1, 1} with their 
expectations gives 

S([0 ! l),Pr{-l,l})= J B([0,l),[-l,l]). 

Now let := e x / (e x + e~ x ). Applying Lemma lXTl we may find a sequence (fk)k 
in B([0,1), [-1,1]) such that 

(<f>(X k )) k l = (f k (U)) k when[/~U[0,l). (9) 
To finish, let 7 be the atomic random measure 

7 = (10) 

fc>i 

so the randomness of 7 is derived from the random choice of the weights v k . 

Clearly one could find many other ways to convert a weighting scheme into an 
ERM, but this translation is appropriate because it gives the correct correspondence 
between upper-bound formulae for the free energy to be recalled below. 
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On the other hand, in ifTTl Panchenko does introduce a family of limit objects, 
closely related to our use of limiting ERMs. Given the random Gibbs measure 7^,at 
on { — 1,1}^, he draws independent replicas a 1 , a 2 , ... from it and then considers the 
joint distribution of the whole (N x oo)-indexed, { — 1, l}-valued random array 

K) Kn<N,l>\- 

Whereas we used these replicas to form an empirical measure which is an ERM, 
Panchenko chooses an arbitrary extension of this to a two-dimensional infinite random 
array. Letting TV — > oo, if one considers a subsequence of the 77V for which these 
joint distrubutions converge, then in the limit one obtains a random { — 1, l}-valued 
array which is row-column exchangeable. Applying Theorem 13.31 this array has the 
same law as 

(cr(C/,C/„,Vi,^)) M > 1 

for some measurable function a : [0, l) 4 — > { — 1,1}, where U, U n for n > 1, Vf, for 
t > 1 and Wru for n,£ > 1 are i.i.d. - U[0, 1). 

Panchenko then uses a itself as his limit object for the sequence (ji3,n)n- The 
equivalence between this formalism and the use of directing random measures on 
B([0, 1), Pr{ — 1, 1}) is just the equivalence between our Theorem B and Theorem 13. 3 1 
described in Subsection l3.3l above. 

6.3 Formula for the limiting free energy 

A central result of |fT7l is a formula for the asymptotic expected free energy of models 
such as (0 in terms of a functional of the directing functions introduced above: see IfTTl 
Theorem 2]. For the VB model itself the result is as follows. 

Theorem 6.1 (Free energy formula). As N — > 00 one has 

lim Fm = inf 'P(er), 

N >oo a 

where for a : [0, l) 4 — > { — 1,1} we have 

V{a) log 2 + log E< 2 > ( cosh/3 ^ J z <j{W, U, V tl X^ 

i=l 

- E« log E« ( exp JMW, U, Vi,Xi)a(W, U, V[, X'S) , 

i=l 

where: 

• all the r.v.s W, U, Vi, V 2 , V{, V{, X lr X 2 , X[, X' 2 , ...are i.i.d. 
~ U[0, 1), 

• K\ is an independent Poisson r.v. of mean 2a, 

• K2 is an independent Poisson r.v. of mean a, 
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• and the coefficients Ji are chosen independently from the same distribution as 
before, 

and where 

E^ = expectation over W, K±, K2, {Viji, (Yl)i ond (Ji)i 

and 

E^ 2 -* = expectation over U, (Xi)i and (X'^i. 

□ 

If 7 is the random directing measure on B([0, 1), Pr{— 1, 1}) that corresponds to 
a, then the above formula may easily be recast in terms of 7: it is 

io g 2+Eiog/ n^'^t^E^)^ 

Jb ei,...,e Kl =±l i=l i=l 

K 2 K 2 

-Eiog / n/^'i £ ^)/( F -{ £ ^)( ex p^E J ^ £ U 7(d/) ' 

d,..., E K2 =±1 i=l »=1 

■ • -.s'k-2 = ±1 

where 

B = B([0,l),Pr{-l,l}), 

and where E is now the expectation over all the random data 7, Ki, K%, (Viji, {V()i 
and (Jj)j. 

Another elementary (but tedious) calculation shows that under the correspondence (TTOt 
this coincides with the upper-bound expression that appears in [20|: the right-hand side 
of inequality (3.3) in that paper. It is for the sake of this calculation that one uses the 
function $ to define fk in (|9). 

Remark. In |17| Panchenko also shows that the quantity above is unchanged if one 
instead takes the infimum only over those directing functions a that satisfy an analog 
of the Aizenman-Contucci stability under cavity dynamics. This modification could 
also easily be formulated in terms of random directing functions, but we omit it for the 
sake of brevity. <\ 



6.4 The analog of ultrametricity 

After the general formalism of Section 3 of lEOl . Sections 4 and 5 of that paper propose 
a special class of weighting scheme objects that correspond to the physicists' notion 
of 'replica-symmetry breaking', and conjecture that these give the correct expression 
for the limiting free energy. Following the prescriptions of the preceding subsections, 
we can translate this conjecture into a proposal for a class of limiting random directing 
measures which adapt the classical Parisi ultrametricity ansatz [19| to the setting of 
dilute models. As before, the necessary calculations are simple but tedious, so we omit 
the details. Some discussion along these lines is given in [ 17 1 for the SK model, rather 
than for dilute models. 
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The key objects seem to be the following. Suppose that T is a discrete rooted tree 
with all leaves at a fixed finite distance from the root. (The discussion that follows can 
certainly be extended to more general trees, but we omit that here.) Let * be the root 
and dT the set of leaves. Also, let E be the Borel u-algebra of [0, 1). We formulate the 
following on [0, 1), but it clearly makes sense on any probability space. 

Definition 6.2. A branching filtration on ([0, 1), S, Leb) indexed by T is a family of 
a-subalgebras (E t ) tS T such that 

• t < t' => E t c S t /; 

• for any to, ... , t m , the a-algebra E t[) is conditionally independent from E tl V 
■ • • V E tm over E s where s — (t A ti) V (to A tz) V • • • V (t A f m ), the closest 
vertex ofT to t which is a common ancestor oft and some other t j. 

By analogy with ordinary filtrations, the branching filtration is complete if every E t is 
complete for Lebesgue measure. 

Given a branching filtration E = (E t ) te T, « branchingale adapted to E is a 
family of integrable ^-valued functions (ft)teT on [0, 1) such that 

• ft is Tit-measurable; 



Observe that in this case every root-leaf path *u 1 u 2 ■ ■ ■ Vd gives a martingale (/* , f vi , . . . , f Vr ) 
adapted to the filtration (S*, E^j , . . . , E„ d ); we call the branchingale homogeneous 
if every root-leaf path gives a martingale with the same distribution. 

Sometimes we refer to the whole collection (/{, E t )i S x as a branchingale. 

Remark. Of course, stochastic processes indexed by trees have been studied before, but 
I have not been able to find a reference for precisely this notion. Much of the literature 
concerns tree-indexed Markov processes, as in 0], but I do not see why the r.v.s /„ that 
we will use should have the Markov property. < 

Definition 6.3. A subset Y C B([0, 1), [— 1, 1]) is hierarchically distributed if it 

equals {/„ : v G dT} for some homogeneous branchingale (/ t , E t ) te x. The min- 
imal depth ofT in such a representation is the depth of the set Y. 

Now a simple calculation shows that under the correspondence ( fTOb . the special 
weighting schemes used to formulate the r-step replica-symmetry breaking bound 
in ||20l Section 5] correspond to random measures 7 which are a.s. supported on heirar- 
chically distributed sets of depth r, and with the weights given by a Derrida-Ruelle 
probability cascade that follows the indexing tree. 

To be more specific, in their work they consider r.v.s X t indexed by the leaves t 
of a tree T of depth r and infinite branching, and specify their joint distribution by 
constructing a larger family of random variables 



• t < a 



ft = E(f t > I S t ). 



(„(0) n (i) „(2) (r-i) 

VI 1 'It! J '/tii 2 > • • • ' '/tita— i r _ 



1' 




indexed by all downwards paths from the root in T, where: 
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• 77ti - t r — Xt r for each leaf t r £ dT, 

(s) 

• for a shorter path t\t2 • • • t s , < s < r — 1, the r.v. takes values in the 
space 

Pr(Pr(- • -Pr(M))), 



• and for each t\ti ■ ■ ■ t s with s < r — 1, the r.v.s ( indexed by all the chil- 

dren t of i s are chosen independently from rflj. ... t , and similarly the random 
variables at all further children along distinct ancestral lines are conditionally 
independent. 

Such a structure arises from a homogeneous branchingale (/ t , S t ) tS T for which < 
ft < 1 a.s. as follows. Let rfc^..^ be the conditional distribution of $ o / tj , 
on St r _j for any child i r of t r _i, where = e x /(e x + e~ x ) as before, and the 

condition < ft r < 1 ensures that this composition is defined a.s.. Now let ry^.T.j _ 
be the conditional distribution of on St r _ 2 , and so on. These are then related 

(s) 

to the functions f t themselves in that f ts is obtained from by applying $ and 

(s) 

then taking barycentres i — s times. If one starts instead from the r.v.s rfc ... t as above, 
another simple (but lengthy) iterated appeal to Lemma |3~T| produces a homogeneous 
branchingale that gives rise to it. 

Thus, the natural analog of the Parisi ultrametricity ansatz for the Viana-Bray 
model seems to be that in the infimum of Theorem 16.11 if one formulates the right- 
hand side in terms of directing random measures, it is enough to consider direct- 
ing random measures that are a.s. supported on heirarchical ly distributed subsets of 
B([0,l),Pr{-l,l}). 
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